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REMARKS 



Applicants amend claims 1,6, 13, 19, 21 and 23 and add new claim 25. Support for the 
amendments and the new claim is provided throughout the specification. Thus, no new matter is 
added. As discussed in detail below, application is in condition for allowance. Hence, 
reconsideration and allowance are respectfully requested. 

Rejections Under 35 U.S.C. 112 

In response to rejections of claims 8, it is amended to depend on claim 7, which recites an 
imager, rather than claim 6, thereby providing sufficient antecedent basis for recitation of the 
imager. 

Further, claim 21 is amended to recite "a display," thereby eliminating the need for 
antecedent basis. Accordingly, the rejections of claim 21, as well as claim 22, which was 
rejected due to its dependence on claim 21, are overcome. 

Rejections Under 35 U.S.C. 103 

The Office Action rejects claims 1-16 and 23 as being obvious in view of the combined 
teachings of U.S. Patent No. Re 36,207 of Zimmerman and U.S. Patent No. 5,077,609 of 
Manelphe. 

Broadly speaking, the present invention provides apparatus and methods for generating a 
perspective-corrected view of a portion of a wide-angle image. The claimed device employs a 
fish-eye lens to project a radially-distorted image onto an image plane. The device includes a 
processor with a first module that displays the distorted image, and a second module that 
displays a selected portion of the image in perspective-corrected form. 

Importantly, and unlike prior art systems, the processor achieves perspective correction 
of the distorted portion from a vantage point offset from the center of a "hemispherical view" 
corresponding to the wide-angle image. This allows enhanced flexibility for viewing 
perspective-corrected views of the wide-angle image, even for viewing the full 180-degree field 
in perspective-corrected format. 
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Neither Zimmerman nor Manelphe teaches or suggests a processor that generates a 
perspective-corrected view of an image obtained by a fish-eye lens from a vantage point offset 
from the center of a hemisphere associated with the fish lens'field of view. More particularly, 
Zimmerman describes a device for multi-directional image viewing that provides pan-and-tilt, 
rotation, and magnification without utilizing any moving parts. The Zimmerman device includes 
a fish-eye lens that generates a distorted image of a hemispherical field of view, which is 
captured into an electronic buffer memory. A portion of the captured image corresponding to a 
region of interest is then transformed into a perspective-corrected image. This transformation is, 
however, accomplished by processing the distorted image based on an angle subtending the 
region of interest from a vantage point located at the center of the hemispherical field. 

In contrast, the processor in the claimed device generates a perspective-corrected view of 
the distorted image from a vantage point offset from the center of a viewing hemisphere by a 
selected distance. The term "viewing hemisphere" is utilized herein as a short hand to refer to 
any wide-angle viewing field that has radial distortions in at least one dimension. 

Hence, the image processing method of the invention is fundamentally distinct from that 
of Zimmerman, and results in significant advantages not provided by Zimmerman. In particular, 
Applicants explain that conventional methods for undistorting a portion of a fish-eye image, such 
as those described by Zimmerman, suffer from a number of shortcomings. For example, these 
methods can only be utilized for removing distortions from image portions corresponding to a 
small field of view, and will break down when applied to image portions corresponding to fields 
of view of tens of degrees, thus considerably diminishing the benefits of panoramic observation 
of a scene. See, e.g., specification, page 1, paragraph 4. For example, an "undistorted image 
obtained by utilizing such a conventional method exhibits progressively less resolution at points 
further away from the center of the image." In fact, "application of such conventional methods 
for undistorting even modest portions of a fish-eye image results in severe loss of resolution at 
points of the corrected image corresponding to edges of the undistorted image." See 
specification, page 18, paragraph 2. 
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In contrast, the transformation of a distorted fish-eye image from a vantage point offset 
from the center of the hemispherical field of view, as recited in claim 1, "affords enhanced 
flexibility for viewing portions of the fish-eye image throughout the full 180-degree field of view 
while compensating for distortions introduced by the fish-eye lens." For example, a comparison 
of FIGURE 9B with FIGURE 9C, presented in the specification, illustrates that the application of 
the methods of the invention to a distorted wide-angle image results in a perspective-corrected 
image that is "significantly enhanced in fidelity relative to an undistorted image obtained by 
employing conventional techniques." See specification, page 19. 



The shortcomings of the Zimmerman are not bridged by Manelphe, which is directed to a 
wide-angle camera in which a wide-field image of scene is displayed on a first monitor and a 
small-field image corresponding to a section of the wide-field image is displayed on a second 
monitor. More specifically, unlike the claimed invention, Manelphe does not teach correcting a 
portion of the wide-angle image for perspective distortions from a vantage point offset from the 
center of a hemisphere corresponding to that image. 

Hence, claim 1 distinguishes patentably over the combined teachings of the cited 
references. Further, claims 2-9 depend either directly or indirectly on amended claim 1, and 
hence are also patentable. 



Claim 10, as amended, depends on claim 1, and further recites that the processor 
generates the perspective-corrected view by mapping a point (u,v) on an undistorted image plane 
corresponding to a perspective-corrected portion of a distorted image to a point (x,y) on a plane 
corresponding to the distorted image, where the mapping is provided by the following equations: 

x = R(p 0 /(PI/2)) cos(do) 

y = R(p 0 /(PI/2)) sin(do) 

t = [Dd + sqrt(D 2 d 2 - (u 2 + v 2 + d 2 ) (D 2 - 1))] / (u 2 + v 2 + d 2 ) 
p 0 = arctan(-D cos(P) + d t cos(P) - v t sin(p), 1) 
5 0 = arctan(-D sin(P) + d t sin(P) + vt cos(p), u t) + 5 
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where p and 8 are the zenith and azimuthal angles corresponding to the center of the 
undistorted image, Po and 5o are the zenith and azimuthal angles corresponding to a pixel (u,v) in 
the undistorted image, d is a magnification factor, D is a level of undistortion, and R is the radius 
of the fisheye image. 

The equations presented in claim 10 are fundamentally distinct from those of Zimmerman 
in that these equations utilize an offset, namely, parameter D, relative to the center of 
hemispherical field of view that Zimmerman does not teach or suggest. As discussed in detail 
above, such an offset, though rendering the equations algebraically more complex, provides 
significant advantages. Hence, claim 10 is also patentable over the combination of the two cited 
patents. Moreover, claims 1 1 and 12 depend either directly or indirectly on claim 10, and are 
also patentable. 

Independent claim 13, as amended, recites a device for providing a perspective-corrected 
view of at least a portion of a wide-angle image that includes an image-capturing device having a 
fish-eye lens for generating the wide-angle image, and a processor coupled to the image 
capturing device that receives the image. The processor displays the image and an associated 
graphical object for selecting a portion thereof. The processor responds to the selection of a 
portion of the image by generating a perspective-corrected view of that portion and replacing that 
image portion with the corrected view in the display of the image. 

Zimmerman does not teach or suggest replacing a portion of a fish-eye image with its 
perspective-corrected version within a single display. Further, in Manelphe's device, a wide- 
field image is displayed on a first monitor and a small-field image corresponding to a selected 
portion of the wide-field image, possibly with a degree of enlargement, is displayed on a second 
monitor. That is, Manelphe does not replace the selected portion with an enlarged version of that 
portion within the original display, but rather displays the enlarged portion on a separate monitor. 

In contrast, claim 13 recites that a selected portion of the distorted image is replaced with 
its perspective corrected version in the original display of the image. This provides a number of 
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advantages. For example, a "user can view at a glance not only a perspective-corrected portion 
of interest in the distorted image but also the relationship of this portion relative to the remainder 
of the distorted image." 

Hence, claim 13, and claims 14 and 15 dependent on claim 13, distinguish patentably 
over the combined teachings of Zimmerman and Manelphe. Moreover, claim 16 that depends on 
claim 15 not only incorporates the patentable features of this claim, but it further recites 
additional patentable features. For example, it recites that the processor corrects the selected 
portion of the image for perspective distortions in accord with the selected magnification and 
angle for viewing a portion of a hemispherical field of view corresponding to the selected image 
portion from a vantage point offset from a center of the hemisphere. As discussed in detail 
above, utilizing a vantage point offset from the center of hemispherical field of view is not taught 
by Zimmerma. Nor does Manelphe teach such a feature. Hence claim 16 is also patentable over 
the cited references. 



Independent claims 23 recites a device for imaging a field of view that includes an 
image-capturing device having a fish-eye lens for acquiring a fish-eye image of the field of view, 
a processor in communication with the image-capturing device, and a display coupled to the 
processor for presenting the fish-eye image and a perspective-corrected view of a portion thereof. 
The device further includes a graphical object presented on the display in association with a 
portion of the fish-eye image. The processor effects the presentation of the fish-eye image on the 
display, and further generates a perspective-corrected view of the portion of the fish-eye image 
associated with the graphical object. It also causes display of the perspective-corrected view 
within an area circumscribed by the graphical object. This allows not only viewing a 
perspective-corrected version of the selected area, but also discerning its juxtaposition relative to 
the other portions of the underlying fish-eye image. 

Neither Zimmerman nor Manelphe presents a perspective-corrected view of a portion of a 
fish-eye image within an area circumscribed by a graphical object associated with the image on a 
single display. In fact, as discussed above, Manelphe utilizes a separate display for presenting a 
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perspective-corrected view of a portion of a wide-field image, and Zimmerman simply provides 
perspective-corrected view of regions of interest in a fish-eye image without simultaneously 
presenting the underlying image and the relationship of a corrected region relative to the other 
parts of the image. 

Thus, claim 23 is patentable over the cited art. 

Rejections Under 35 U.S.C. 102 And Further Rejections Under 103 

The Office Action rejects claim 17 as being anticipated by Zimmerman. 

Claim 17 recites a device for providing a perspective-corrected view of at least a portion 
of a wide angle image, which includes an image-capturing device having a fish-eye lens for 
generating the wide angle image, and a processor in communication with the image-capturing 
device. The processor corrects at least a portion of the image for distortions introduced by the 
fish-eye lens by mapping a point (u,v) of an undistorted image plane, corresponding to a 
perspective-corrected view of the image portion, to a point (x,y) of a distorted image plane, 
corresponding to the uncorrected image, in accord with an angle for viewing a section of the 
hemisphere corresponding to the image portion from a vantage point offset from a center of the 
hemisphere and a distance between the vantage point and the center of the hemisphere. 

The arguments presented above apply with equal force to establish that claim 17 
distinguishes patentably over Zimmerman. In particular, as discussed in detail above, 
Zimmerman does not teach a processor that corrects at least a portion of a fish-eye image from a 
vantage point offset from a center of a hemisphere corresponding to the fish-eye image and a 
distance between the vantage point and the hemisphere center. As also explained above, 
utilizing such an offset vantage point provides distinct advantages, e.g., it allows perspective 
correction of large portions, or even the entirety, of the fish-eye image without introducing 
artifacts. 
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In Paragraph 28, the Office Action rejects claims 18 and 24 as being obvious over 
Zimmerman. 

As an initial matter, claim 18 depends on claim 17, and hence incorporates its patentable 
features. As discussed above, claim 17 distinguishes patentably over Zimmerman. Hence, 
similarly, claim 18 is patentable over this reference. Additionally, claim 18 recites that the 
processor employs the following equations to effect the mapping between the point (u,v) and the 
point (x,y): 

x = R(p 0 /(PI/2)) cos(do) 

y = R(p 0 /(PI/2))sin(do) 

t = [D d + sqrt(D 2 d 2 - (u 2 + v 2 + d 2 ) (D 2 - 1))] / (u 2 + v 2 + d 2 ) 
Po = arctan(-D cos(P) + d t cos(P) - v t sin(p), 1) 
8 0 = arctan(-D sin(p) + d t sin(p) + vt cos(p), u t) + 8 

where P and 5 are the zenith and azimuthal angles corresponding to the center of the 
undistorted image, p 0 and 5 0 are the zenith and azimuthal angles corresponding to a pixel 
(u,v) in the undistorted image, d is the magnification factor, D is the level of undistortion, 
and R is the radius of the fisheye image. 

As discussed above, these equations are distinct from those disclosed by Zimmerman in 
that they include an offset factor, i.e., the parameter D, that Zimmerman does not teach or 
suggest. 

Claim 24 recites a method for generating a perspective-corrected view of a portion of a 
fish-eye image by mapping a point (u,v) on an undistorted image plane corresponding to a 
perspective view of the image portion to a point (x,y) on a distorted image plane corresponding 
to the fish-eye image according to the following equations: 

x = R(p 0 /(PI/2)) cos(do)' 

y = R(Po/(PI/2)) sin(do) 

t = [D d + sqrt(D 2 d 2 - (u 2 + v 2 + d 2 ) (D 2 - 1))] / (u 2 + v 2 + d 2 ) 
Po = arctan(-D cos(P) + d t cos(P) - v t sin(p), 1) 
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5 0 = arctan(-D sin(P) + d t sin(P) + vt cos(P), u t) + 5 

where P and 5 are the zenith and azimuthal angles corresponding to the center of the 
undistorted image, Po and 5o are the zenith and azimuthal angles corresponding to a pixel 
(u,v) in the undistorted image, d is the magnification factor, D is the level of undistortion, 
and R is the radius of the fisheye image 



The arguments presented above with respect to claim 18 apply to establish that claim 24 
is patentable, as well. In particular, Zimmerman fails to teach or suggest a material feature of the 
equations recited in this claim, namely, an offset D that corresponds to level of undistortion. 

In Paragraph 31, the Office Action rejects claims 19 and 20 as being obvious over 
Zimmerman in view of U.S. Patent No. 5,489,940 of Richardson. 



Independent claim 19 recites a device for imaging a field of view, which includes an 
image-capturing device having a fish-eye lens for generating a wide angle image of the field of 
view, and a processor coupled to the image-capturing device to receive the image. The processor 
can cause display of the wide-angle image and can further select a portion of the image based on 
a pre-programmed set of rules. Upon such selection, the processor generates a perspective- 
corrected view of the selected portion from a vantage point offset from the center of a viewing 
hemisphere and displays it. 



As discussed above, Zimmerman does not teach generating a perspective-corrected view 
of a portion of a fish-eye lens from a vantage point offset from the center of a viewing 
hemisphere. Nor does it teach selecting a portion of a fish-eye image for perspective correction 
based on a pre-programmed selection rule. Moroever, Richardson does not bridge the 
shortcomings of Zimmerman to render the claimed invention obvious. Richardson is directed to 
an electronic imaging system in which a wide-angle lens projects an image of a scene onto a 
two-dimensional array of non-linearly distributed imaging elements. The non-linear distribution 
is designed to correct distortions introduced by the wide-angle lens. Thus, Richardson does not 
correct a distorted fish-eye lens image by applying a corrective procedure to data corresponding 
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to a distorted image, let alone the procedure described in the claimed invention. Rather, it relies 
on hardware, i.e., non-linear design of imaging elements, for perspective correction. 

Thus, claim 19, and claim 20 that depends on claim 19, distinguish over the combined 
teachings of Zimmerman and Richardson. 

In Paragraph 34, the Office Action rejects claims 21 and 22 as being obvious over 
Zimmerman in view of Richardson. 

The arguments presented above in connection with claim 19 apply with equal force to 
establish that, similar to claim 19 on which these claims indirectly depend, they are also 
patentable over combined Zimmerman and Richardson. 



New Claim 



New claim 25 recites an imaging device that includes an image-capturing device having a 
wide-angle lens for generating an image of at least a portion of a viewing field associated with 
the lens, and a processor coupled to said image-capturing device for generating perspective 
correction of at least a portion of said image from a vantage point offset from a center of the 
viewing field. 

As discussed in detail above, none of the cited art teaches or suggests an imaging device 
in which a processor corrects a portion of a wide-angle image from a vantage point offset from 
the center of the viewing field. Hence, new claim 25 is patentable over the cited art. 
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CONCLUSION 

In view of the above amendments and remarks, Applicants respectfully request 
reconsideration and allowance of the application. Applicant invites the Examiner to call the 
undersigned at (617) 439-2514 if there are any remaining questions. 



Dated: 



Respectful ly submitted, 





Reza Mollaaghababa 
Regi^ration No.: 43,810 

ITER MCCLENNEN & FISH LLP 
World Trade Center West 
155 Seaport Boulevard 
Boston, Massachusetts 02210-2604 
(617) 439-2000 
(617) 310-9000 (Fax) 
Attorney for Applicant 
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